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PRIORITY INFORMATION 

This application claims priority from US Provisional Patent Application, Serial 
Number 60/261,023, filed on January 11, 2001; the entire contents of which are hereby 
incorporated by reference. 

FIELD OF THE PRESENT INVENTION 

The present invention is directed to controlling sensitivity, on a pixel-by-pixel 
basis, of a digital imager. More specifically, the present invention is directed to 
adaptively controlling sensitivity, on a pixel-by-pixel basis, of a digital imager to expand 
a dynamic range of the digital imager. 

BACKGROUND OF THE PRESENT INVENTION 

Conventionally, digital imagers, as noted above, are characterized by a linear 
voltage-to-light response, or transfer function; that is, the digital imager output voltage is 
approximately linearly related to light incident on the digital imager. Specifically, the 
output voltage transfer function is linearly proportional to the intensity of the light 
incident upon the digital imager. This linear transfer function can be characterized by a 
dynamic range. The dynamic range is defined as the difference between the highest 
detectable illumination intensity of a scene to the lowest detectable illumination intensity 
of the scene. It is well understood that the dynamic range of the transfer function sets the 
overall dynamic range of the digital imager. If the dynamic range of a scene exceeds the 
dynamic range of a digital imager, portions of the scene will saturate the digital imager 
and appear either completely black or completely white. This can be problematic for 
imaging large dynamic range scenes, such as outdoor scenes. 

A conventional digital imager includes a plurality of photosensitive elements 
commonly referred to as pixels. The physical realization of the pixels is either a plurality 
of phototransistors or a plurality of photodiodes functioning as light-detecting elements. 
In operation, the pixel is first reset with a reset voltage that places an electronic charge 



across the capacitance associated with the diode. Electronic charge produced by, for 
example, a photodiode, when exposed to illumination, causes charge of the diode 
capacitance to dissipate in proportion to the incident illumination intensity of the scene. 
At the end of an exposure period, the change in diode capacitance charge is detected and 
the photodiode is reset. The amount of light detected by the photodiode is computed as 
the difference between the reset voltage and the voltage corresponding to the final diode 
capacitance charge. 

The sensitivity of a digital imager, such as that provided in a digital still or video 
camera, can in general be defined by the maximum and minimum light intensities of a 
scene that can be imaged by the digital imager. The light or illumination intensity of a 
scene captured by a pixel of a digital imager is a value related to an output voltage or 
amount of charge measured at the pixel, photosensitive element. 

There are various situations that can limit the sensitivity of a digital imager. One 
example is pixel saturation. Pixel saturation is realized when a photosensitive element, 
pixel, becomes depleted of charge such that no amount of additional incident light will 
change the output voltage, thus pixel saturation acts to limit the maximum light intensity 
of the scene that can be discretely imaged. In other words, if the saturation of a pixel is 
an illumination intensity level of 100, all illumination intensity levels of the scene that are 
above an illumination intensity level of 1 00 are represented as an illumination intensity 
level of 100 in the imaged scene, and thus, these illumination intensity levels of the scene 
that are above an illumination intensity level of 1 00 are not discretely imaged or are not 
discernible because notwithstanding the actual illumination intensity level, above 100, the 
imaged illumination intensity level is 100. 

Another example is pixel noise. Examples of sources of pixel noise are 
quantization in the conversion of the analog voltage to a digital code, integrated thermal 
charge production, and voltage reset variation when a pixel is not sampled using a true 
correlated double sample. Pixel noise sets the minimum detectable illumination intensity. 
Since the human eye has a logarithmic response to intensity, in digital imaging, it is 
usually noise in the lower intensities that is of concern. 

Both pixel saturation and pixel noise are inherent properties of a digital imager. 
So, these properties limit the physical sensitivity of a digital imager such that a scene, 



having a wide range of light (illumination) intensities, cannot be imaged properly; the 
higher light (illumination) intensities of the scene are washed out in the imaged scene due 
to pixel saturation, or the low light (illumination) intensities of the scene become less 
resolvable due to pixel noise. 
5 Conventionally, methods have been proposed to nullify these physical limitations 

of a digital imager, such as the manipulation or adjustment of the charge integration 
function of the pixel, also known as the transfer control function of the digital imager. 
Charge integration function manipulation or transfer control function manipulation has, 
conventionally, been realized through the changing of an integration time, T mt , for the 

1 0 digital imager. Changing the integration time, T mt , changes the start time of the transfer 
control function or charge integration period. In other words, changing the integration 
time is a form of electronic-irising or exposure control, analogous to controlling the speed 
of a shutter on an analog camera. An example of conventional integration time 
manipulation is illustrated in Figures 1 and 2. 

1 5 Figure 1 shows a linear transfer control function wherein an integration time, T int , 

is near a maximum integration time. Integration time, T int , is the time that a control 
signal 10 is not set at a reset level. When the control signal 10 is not at a reset value 
during a frame period, F, the digital imager causes charge to be transferred or collected 
from a pixel. As shown in Figure 1, the control signal 10 is initially at a reset level (at the 

20 beginning of the frame period, F); however, the control signal 10 changes to another 
level, in this case a collect level, at a point in time, within the frame period, that is equal 
to a difference between the frame period, F, and the integration time, T int . (In Figure 1 , 
the difference is very small.) The control signal 10 changes back to the reset level at the 
end of the frame period, F. 

25 On the other hand, Figure 2 shows a linear transfer control function wherein the 

integration time, T mt , is decreased. As noted above, integration time, Ti m , is the time that 
a control signal 10 is not set at a reset level. When the control signal 10 is not at a reset 
value during a frame period, F, the digital imager causes charge to be transferred or 
collected from a pixel As shown in Figure 2, the control signal 10 is initially at a reset 

30 level (at the beginning of the frame period, F); however, the control signal 10 changes to 
another level, in this case a collect level, at a point in time, within the frame period, that 
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is equal to a difference between the frame period, F, and the integration time, T int . (In 
Figure 2, the difference is approximately equal to half the frame period, F.) The control 
signal 10 changes back to the reset level at the end of the frame period, F. 

In other words, Figure 2 shows a situation where the control signal 10 is set at a 
level that causes charge to be transferred or collected from the pixel for a shorter duration 
of time than shown in Figure 1 . By decreasing the integration time, the low illumination 
intensity levels of the scene being imaged are de-emphasized because not enough time is 
given to for the charge associated with the low illumination intensity levels of the scene 
to accumulate because the low illumination intensity level does produce charge as 
quickly as the high illumination intensity levels. 

Another way of looking at the relationship of the integration time, T int , of the 
digital imager to the illumination intensity level of the imaged scene is shown in Figure 3. 
In Figure 3, curve A represents a first mapping of pixel output voltage versus 
illumination intensity level of the imaged scene using a first integration time, Tl int . 
Curve B shows a second mapping of pixel output voltage versus illumination intensity 
level of the imaged scene using a second integration time, T2 ]nt . In Figure 3, the second 
integration time, T2 mt , is smaller than the first integration time, Tl mt . Thus, as illustrated 
in Figure 3, as the integration time, T mt , is increased, the slope of the map of pixel output 
voltage versus illumination intensity level of the imaged scene is increased, thereby 
emphasizing the lower illumination intensity levels of the imaged scene. 

Another conventional way of manipulating the transfer control function is to use a 
stepped or piecewise discrete-time transfer control function. By using a stepped or 
piecewise discrete transfer control function, the mapping of the pixel output voltage 
versus illumination intensity level of the imaged scene can be modified to enable a wider 
range of possible illumination intensity levels of the imaged scene before saturation, 
while emphasizing low illumination intensity levels of the imaged scene. 

An example of a conventional stepped or piecewise discrete-time transfer control 
function is illustrated in Figure 4. In Figure 4, the integration time, T int , is the same value 
as illustrated in Figure 1. However, unlike Figure 1, the level of the control signal 10, in 
Figure 4, is stepped down in a discrete fashion from a reset level to a collect value. The 
control signal 10 acts as a barrier to charge production or accumulation. At the reset 



level, a complete barrier is realized wherein no charge is produced or accumulated, 
notwithstanding the illumination intensity level of the light incident upon the pixel In 
other words, the pixel is effectively turned off. As the control signal 10, is stepped down 
from a reset level, the effective barrier is lower, thus allowing a rate of production or 
accumulation of charge to effectively realize a gradual increase. 

Specifically, as illustrated in Figure 4, over the course of an exposure period, the 
control signal 10 is applied to the pixel or photodiode capacitance to control charge 
dissipation from the capacitance. The control signal 10 is typically decreased from the 
starting pixel reset voltage value to, e.g., electrical ground or a collect level, with each 
control signal 10 value at a given time during the exposure period setting the maximum 
charge dissipation of the photodiode. This control signal 10 decrease acts to increase the 
photodiode charge dissipation capability, whereby the pixel can accommodate a higher 
illumination intensity before saturating, and the dynamic range of the pixel is thusly 
increased. This charge dissipation control produces a nonlinear pixel output voltage to 
imaged scene illumination intensity level mapping and a correspondingly expanded 
dynamic range of the pixel and the imager. This is seen in Figure 5. 

Using a step or piecewise discrete-time transfer control function, as illustrated in 
Figure 4, a mapping of the pixel output voltage versus illumination intensity level of the 
imaged scene having a wider range, as illustrated in Figure 5, is produced. In Figure 5, 
curve A shows a mapping of the pixel output voltage versus illumination intensity level 
of the imaged scene when a linear control transfer function, as illustrated in Figure 1 , is 
used, whereas curve B shows a mapping of the pixel output voltage versus illumination 
intensity level of the imaged scene when a stepped or piecewise discrete-time transfer 
control function, as illustrated in Figure 4, is used. As illustrated in Figure 5, curve B has 
a wider range of illumination intensity levels of the imaged scene than curve A. 

Although the physical limitations of digital imagers have been compensated for 
through integration time adjustment and transfer control function (resulting in the 
production of the control signal 10) manipulation, these conventional approaches still fail 
to compensate for the digital imager's physical limitations in an optimal way. 

For example, the conventional processes fail to realize an illumination intensity 
level of the imaged scene range in which no saturated pixels exist. More specifically, as 



illustrated in Figure 6, a histogram of actual illumination intensity levels of a scene to be 
imaged may include illumination intensity levels from 20 to 200 when the image average 
is 50; however, due to pixel saturation, illumination intensity levels of the scene above 
the 100 illumination intensity level will not be represented by the digital imager, as 
illustrated Figure 7, because these higher illumination intensity levels of the scene are all 
bucketed at the 100 illumination intensity level by the digital imager due to pixel 
saturation, thereby washing out the higher illumination intensity levels of the imaged 
scene. 

Using the conventional processes described above to compensate for the saturated 
pixels by reducing the integration time, a histogram of the illumination intensity levels of 
the imaged scene is realized as illustrated in Figure 8. In Figure 8, the histogram of the 
imaged scene is scaled by a factor of 0.5 such that the image illumination intensity 
average changes from 50 to 25. Using this conventional approach, a greater portion of 
the higher illumination intensity levels of the scene can be represented; however, the 
image is unacceptably dark, the quantization noise of the entire image has been cut in 
half, and the signal may also have been reduced by a factor of 2 while other noise sources 
stay fixed in magnitude (such as reset noise). 

SUMMARY OF THE PRESENT INVENTION 

A first aspect of the present invention is a method of adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The method determines a 
number of pixels of image data having illumination intensity levels within a first defined 
range of illumination intensity levels; determines an illumination intensity level mapping 
function based upon the determined number of pixels within the first defined range of 
illumination intensity levels; determines a transfer control function based on the 
determined illumination intensity level mapping function; and imposes the determined 
transfer control function upon a pixel of the digital imager, 

A second aspect of the present invention is a method of adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The method determines a 
plurality of numbers of pixels, each determined number of pixels being a number of 
pixels within an associated defined range of illumination intensity levels; determines a 



plurality of illumination intensity level mapping functions, each determined illumination 
intensity level mapping function corresponding to one defined range of illumination 
intensity levels, each illumination intensity level mapping function being determined 
based upon the determined number of pixels within the corresponding defined range of 
illumination intensity levels; determines a transfer control function based on the plurality 
of determined illumination intensity level mapping functions; and imposes the 
determined transfer control function upon a pixel of the digital imager. 

A third aspect of the present invention is a method of adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The method determines a 
number of saturated pixels; selects a first illumination intensity level mapping function 
when the determined number of saturated pixels is above a first threshold; determines an 
number of pixels having illumination intensity levels within a defined range of values; 
selects a second illumination intensity level mapping function when the determined 
number of pixels is below a second threshold; determines a transfer control function 
based on the selected illumination intensity level mapping function; and imposes the 
determined transfer control function upon a pixel of the digital imager. 

A fourth aspect of the present invention is a method of adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The method determines a 
number of pixels of image data having illumination intensity levels within a first defined 
range of illumination intensity levels, the first defined range of illumination intensity 
levels including an illumination intensity level corresponding to a pixel saturation value; 
determines an illumination intensity level mapping function based upon the determined 
number of pixels within the first defined range of illumination intensity levels; 
determines a number of pixels having illumination intensity levels within a second 
defined range of illumination intensity levels, the second defined range of illumination 
intensity levels including an illumination intensity level corresponding to a minimum 
illumination intensity level; determines an integration time based upon the determined 
number of pixels having illumination intensity levels within the second defined range of 
illumination intensity levels; determines a transfer control function based on the 
determined illumination intensity level mapping function and the determined integration 



time; and imposes the determined transfer control function upon a pixel of the digital 
imager. 

A fifth aspect of the present invention is a method of adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The method selects a first 
illumination intensity level mapping function; determines a first transfer control function 
based on the selected first compression; imposes the determined first transfer control 
function upon a pixel of the digital imager; determines a histogram of illumination 
intensity levels of pixels of image data being generated by the digital imager having the 
determined first transfer control function imposed thereon; determines an illumination 
intensity level maximum, the illumination intensity level maximum representing a 
greatest illumination intensity level for a pixel in a sample forming the histogram; 
determines a second illumination intensity level mapping function, based on the 
determined intensity level maximum, the second illumination intensity level mapping 
function preventing the generation of any saturated pixels and providing a dynamic range 
of image data enabling each level in the histogram to be realized by the digital imager; 
determines a second transfer control function based on the determined second 
illumination intensity level mapping function; and imposes the determined second 
transfer control function upon a pixel of the digital imager. 

A sixth aspect of the present invention is a method for determining transition 
points between a plurality of discrete transfer control functions forming a composite 
transfer control function. The method determines an integration time; determines an 
illumination intensity level mapping function; determines a composite transfer control 
function based on the determined integration time and determined illumination intensity 
level mapping function; and determines a timing of each transition point between a 
plurality of discrete transfer control functions from the determined integration time and 
the determined illumination intensity level mapping function. 

A seventh aspect of the present invention is a system for adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The system includes an 
illumination intensity level mapping controller to determine a number of pixels of image 
data having illumination intensity levels within a first defined range of illumination 
intensity levels and to determine an illumination intensity level mapping function based 




upon the determined number of pixels within the first defined range of illumination 
intensity levels and a transfer control function generation circuit to determine a transfer 
control function based on the determined illumination intensity level mapping function 
and to impose the determined transfer control function upon a pixel of the digital imager. 

An eighth aspect of the present invention is a system for adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The system includes an 
illumination intensity level mapping controller to determine a plurality of number of 
pixels, each determined number of pixels corresponding to one defined range of 
illumination intensity levels, and to determine a plurality of illumination intensity level 
mapping functions, each determined illumination intensity level mapping function 
corresponding to one defined range of illumination intensity levels, each illumination 
intensity level mapping function being determined based upon the determined number of 
pixels within the corresponding defined range of illumination intensity levels and a 
transfer control function generation circuit to determine a transfer control function based 
on the plurality of determined illumination intensity level mapping functions and to 
impose the determined transfer control function upon a pixel of the digital imager. 

A ninth aspect of the present invention is a system for adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The system includes an 
illumination intensity level mapping controller to determine a number of saturated pixels 
and to select a first illumination intensity level mapping function when the determined 
number of saturated pixels is above a first threshold. The illumination intensity level 
mapping controller determines a number of pixels having illumination intensity levels 
within a defined range of values and selects a second illumination intensity level mapping 
function when the determined number of pixels is below a second threshold. The system 
further includes a transfer control function generation circuit to determine a transfer 
control function based on the selected illumination intensity level mapping function and 
to impose the determined transfer control function upon a pixel of the digital imager. 

A tenth aspect of the present invention is a system for adaptively controlling 
sensitivity, on a pixel-by-pixel basis, of a digital imager. The system includes an 
illumination intensity level mapping controller to determine a number of pixels of image 
data having illumination intensity levels within a first defined range of illumination 



intensity levels, the first defined range of illumination intensity levels including an 
illumination intensity level corresponding to a pixel saturation value, and to determine an 
illumination intensity level mapping function based upon the determined number of 
pixels within the first defined range of illumination intensity levels; an exposure 
5 controller to determine a number of pixels having illumination intensity levels within a 
second defined range of illumination intensity levels, the second defined range of 
illumination intensity levels including an illumination intensity level corresponding to a 
minimum illumination intensity level, and to determine an integration time based upon 
the determined number of pixels having illumination intensity levels within the second 

10 defined range of illumination intensity levels; and a transfer control function generation 
circuit to determine a transfer control function based on the determined illumination 
intensity level mapping function and the determined integration time and to impose the 
determined transfer control function upon a pixel of the digital imager. 

An eleventh aspect of the present invention is a system for adaptively controlling 

15 sensitivity, on a pixel-by-pixel basis, of a digital imager. The system includes an 
illumination intensity level mapping controller to select a first illumination intensity level 
mapping function and a transfer control function generation circuit to determine a first 
transfer control function based on the selected first compression and to impose the 
determined first transfer control function upon a pixel of the digital imager. The 

20 illumination intensity level mapping controller determines a histogram of illumination 
intensity levels of pixels of image data being generated by the digital imager having the 
determined first transfer control function imposed thereon and determines an illumination 
intensity level maximum, the illumination intensity level maximum representing a 
greatest illumination intensity level for a pixel in a sample forming the histogram. The 

25 illumination intensity level mapping controller determines a second illumination intensity 
level mapping function, based on the determined intensity level maximum, the second 
illumination intensity level mapping function preventing the generation of any saturated 
pixels and providing a dynamic range of image data enabling each level in the histogram 
to be realized by the digital imager. The transfer control function generation circuit 

30 determines a second transfer control function based on the determined second 
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illumination intensity level mapping function and imposes the determined second transfer 
control function upon a pixel of the digital imager. 

A twelfth aspect of the present invention is for determining transition points 
between a plurality of discrete transfer control functions forming a composite transfer 
control function. The system includes an exposure controller to determine an integration 
time; an illumination intensity level mapping controller to determine an illumination 
intensity level mapping function; and a transfer control function generation circuit to 
determine a composite transfer control function based on the determined integration time 
and determined illumination intensity level mapping function and to determine a timing 
of each transition point between a plurality of discrete transfer control functions from the 
determined integration time and the determined illumination intensity level mapping 
function. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention may take form in various components and arrangements of 
components, and in various steps and arrangements of steps. The drawings are only for 
purposes of illustrating a preferred embodiment and are not to be construed as limiting 
the present invention, wherein: 

Figure 1 shows a linear transfer control function with an integration time near 
maximum integration time; 

Figure 2 shows a linear transfer control function with a decreased integration 

time; 

Figure 3 illustrates mapping of pixel output voltage versus illumination intensity 
of the imaged scene using different integration times; 

Figure 4 illustrates a stepped or piecewise discrete-time transfer control function; 

Figure 5 illustrates mapping of pixel output voltage versus illumination intensity 
of the imaged scene using a linear transfer control function and a stepped or piecewise 
discrete-time transfer control function; 

Figure 6 is a histogram of illumination intensity levels of an actual scene to be 
imaged; 




Figure 7 is a histogram of illumination intensity levels of the imaged scene of 
Figure 6 without conventional corrections; 

Figure 8 is a histogram of illumination intensity levels of the imaged scene of 

Figure 6 using conventional corrections; 

Figure 9 is a histogram, enabled by the present invention, of illumination intensity 
levels of the imaged scene of Figure 6; 

Figure 10 is a block diagram of the adaptive sensitivity, on a pixel-by-pixel basis, 
control system according to the concepts of the present invention; 

Figure 11 is a flowchart showing one implementation of adaptive sensitivity 
control, on a pixel-by-pixel basis, according to the concepts of the present invention; 

Figure 12 is a flowchart showing another implementation of adaptive sensitivity 
control, on a pixel-by-pixel basis, according to the concepts of the present invention; 

Figure 13 shows calculating the timing of the transition points for a transfer 
control function according to the concepts of the present invention; 

Figure 14 shows a desired map of output voltage versus illumination intensity 
level for a scene imaged to produce the transfer control function of Figure 13; 

Figure 15 illustrates a flowchart showing the operations of an illumination 
intensity mapping controller according to another embodiment of the present invention; 
and 

Figure 16 illustrates a flowchart showing the operations of a transfer control 
function generator circuit according to another embodiment of the present invention. 

DETAILED DESCRIPTION OF THE PRESENT INVENTION 

It is desirable to increase the dynamic range of the digital imager such that the 
low illumination intensity levels of the scene are emphasized, but also the high 
illumination intensity levels of the scene are represented and no pixels become saturated 
during a given integration period. Moreover, it is desirable to utilize the entire dynamic 
range of the digital imager when imaging a scene, notwithstanding the actual range of 

illumination intensities of the scene. 

The present invention provides techniques and systems for adjusting the 
sensitivity, on a pixel-by-pixel basis, of a digital imager by adjusting the transfer control 
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function (resulting in the production of the control signal) through determining an 
illumination intensity level mapping function and an integration time characteristic of the 
digital imager in a manner that enables full use of the entire output intensity level range 
of the digital imager without saturation of the digital imager. These techniques can be 
5 applied to a wide range of digital still and video camera configurations. 

In describing the present invention, various terms have been used. In this 
specification, a frame period is the time period needed to read all the pixels of a digital 
imager to produce one frame of an imaged scene. Integration time refers to the time 
period, within an overall frame period, that a digital imager is receiving incident light or 

10 illumination from a scene being imaged and producing a collectable charge or voltage in 
response to incident light or illumination of the scene being imaged. Illumination 
intensity level mapping function refers to the analog to digital conversion relationship 
between an analog signal received from a pixel of the digital imager and a digital signal 
representing the illumination intensity received by the pixel. Output voltage versus 

15 illumination intensity level map is a graphical representation of the illumination intensity 
level mapping function. Transfer control function refers to the controlling of the charge 
dissipation from the pixel of the digital imager during the integration time period through 
timing manipulation and control signal voltage level or value manipulation. Transfer 
control function generation refers to either the actual generation of the control signal that 

20 controls the charge dissipation from the pixel of the digital imager or a signal that is used 
to generate the control signal that controls the charge dissipation from the pixel of the 
digital imager. 

More specifically, the present invention adjusts the integration time and 
manipulates the transfer control function (resulting in the production of the control 

25 signal) to realize a histogram of the imaged scene, as illustrated in Figure 9. In Figure 9, 
using a pixel-by-pixel sensitivity adjustment, the entire histogram is made to fit within 
the range of the sensor such that no saturation results while keeping the image average at 
50 to ensure the image is bright enough to be visually appealing and thus not reducing the 
signal to noise ratio in the pixels of lower illumination intensities where the human eye is 

30 more sensitive to noise. 
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With the techniques of the present invention, the low illumination intensity levels 
of the scene are emphasized, and the high illumination intensity levels of the scene are 
discernable, with no pixels becoming saturated during a given frame period, F. 

The present invention varies the sensitivity of each pixel of a digital imager so 
that if a scene has a wide dynamic range, the digital imager can produce a non-linear 
mapping of the output voltages from the digital imager to digital illumination intensity 
levels or values representing the imaged scene. According to the concepts of the present 
invention, the output voltages representing the higher illumination intensity levels of the 
scene, where the eye is less sensitive to quantization noise, are non-linearly mapped to 
illumination intensity levels such that the resolution of the higher illumination intensity 
levels are compressed. Moreover, the output voltages representing the low illumination 
intensity levels of the scene are amplified to ensure the image is bright enough. The 
results are imaged scenes with appropriate brightness and no saturation. 

The present invention dynamically adjusts the mapping of the output voltages to 
imaged scene illumination intensity levels, thereby continually compressing or 
decompressing the resolution of the high illumination intensity levels of the scene, so as 
to change or unequally distribute the amount of quantization noise for different levels of 
light (illumination) intensity. Under a completely linear mapping, all possible 
illumination intensities of the imaged scene would have the same resolution or amount of 
quantization noise. 

The present invention selectively decreases or compresses the resolution of higher 
illumination intensity levels, by non-linearly adjusting the mapping of the output voltage 
versus illumination intensity level of the imaged scene map, to increase the saturation 
point. The result is that given a scene with a wide dynamic range, the output voltages 
associated with the low illumination intensity level pixels can be amplified enough to 
have the desired resolution and the output voltages associated with the high illumination 
intensity level pixels can be mapped to compress the resolution of the high illumination 
intensity levels to fit within the range of the analog to digital converter associated with 
the digital imager and avoid saturation. To realize this, in accordance the present 
invention, a desirable illumination intensity level mapping is automatically determined 
and the determined illumination intensity level mapping is converted into a transfer 
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control function (resulting in the production of the control signal) for the digital imager. 
The transfer control function (resulting in the production of the control signal) controls 
the production or accumulation of charge in the photosensitive element, thus controlling 
the output voltage of the pixel. 
5 Figure 10 illustrates one embodiment of the present invention that emphasizes the 

low illumination intensity levels of the scene, and discerns the high illumination intensity 
levels of the scene, with no pixels becoming saturated during a given frame period, F. As 
shown in Figure 10, a system for adaptively controlling sensitivity, on a pixel-by-pixel 
basis, of a digital imager includes a digital imager 1, an illumination intensity level 
10 mapping controller 3, an exposure controller 5, and a transfer control function generation 
circuit 7. 

The illumination intensity level mapping controller 3 determines a number of 
pixels of imaged scene having illumination intensity levels within a first defined range of 
illumination intensity levels. The illumination intensity level mapping controller 3 also 

1 5 determines an illumination intensity level mapping function based upon the determined 
number of pixels within the first defined range of illumination intensity levels. 

The exposure controller 5 determines a number of pixels of the imaged scene 
having illumination intensity levels within a second defined range of illumination 
intensity levels and determines an integration time, T mt , based upon the determined 

20 number of pixels having illumination intensity levels within a second defined range of 
illumination intensity levels. 

The transfer control function generation circuit 7 determines a transfer control 
function based on the determined illumination intensity level mapping function and 
integration time, T mt . The transfer control function generation circuit 7 further imposes 

25 the determined transfer control function upon a pixel of the digital imager. 

An example of a transfer control function generation circuit is described in co- 
pending US Patent Application, Patent Serial Number 09/916,822, filed on July 27, 2001, 
entitled "PRECISE MOS IMAGER TRANSFER FUNCTION CONTROL FOR 
EXPANDED DYNAMIC RANGE IMAGING." The entire contents of co-pending US 

30 Patent Application, Patent Serial Number 09/916,822, are hereby incorporated by 
reference. 
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In a preferred embodiment, the digital imager 1 receives the transfer control 
function, and produces a control signal 10 therein in accordance with the received 
transfer control function. 

It is noted that the transfer control function generation circuit 7 could also 
produce the control signal 10 directly and feed it to the digital imager 1, wherein the 
control signal 10 is produced in accordance with the determined transfer control function 
based on the determined integration time, T mt , and determined illumination intensity level 
mapping function. 

The first defined range of illumination intensity levels is a range of illumination 
intensity levels including an illumination intensity level representing pixel saturation, and 
the second defined range of illumination intensity levels is a range of illumination 
intensity levels including an illumination intensity level representing a minimum 
illumination intensity level that may be adjusted for pixel noise or pixel offset. 

The illumination intensity level mapping controller 3 determines a number of 
pixels of image data having illumination intensity levels within a first defined range of 
illumination intensity levels from a frame of pixels of image data created by the digital 
imager, from a partial frame of pixels of image data created by the digital imager, from a 
defined area within a frame of pixels of image data created by the digital imager, or from 
a user-defined area within a frame of pixels of image data created by the digital imager. 

The illumination intensity level mapping function can be either calculated or 
selected from a plurality of pre-specified illumination intensity level mapping functions, 
preferably from eight pre-specified illumination intensity level mapping functions, based 
upon the determined number of pixels within the first defined range of illumination 
intensity levels. Moreover, the transfer control function can be either calculated or 
selected from a plurality of pre-specified transfer control functions based upon the 
determined illumination intensity level mapping function and determined integration 
time, T int . 

Furthermore, the determination of the transfer control function can realized during 
the generation of a second frame of image data that is immediate subsequent to the frame 
of image data being used to make the determination. In this situation, the new transfer 
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control function is imposed upon pixels during the generation of a third frame of image 
data, the third frame being immediate subsequent to the second frame of image data. 

On the other hand, if the determination is being made on only a partial frame, the 
determination of the transfer control function can realized during the generation of a first 

5 frame of image data which is the frame of image data having the partial frame of image. 
In this situation, the new transfer control function is imposed upon pixels during the 
generation of a second frame of image data, the second frame being immediate 
subsequent to the first frame of image data. 

It is further noted that the frame period, F, or other frame parameters can be 

10 adjusted on the fly to enable implementation of the present invention. 

The illumination intensity level mapping function may be determined independent 
of, dependent upon, after, before, or substantially simultaneous with the determination of 
the integration time, T int . Moreover, the illumination intensity level mapping function 
may be a composite function comprising individual discrete functions wherein each 

15 individual discrete illumination intensity level mapping function is determined based 
upon the determination of a number of pixels within a defined range of illumination 
intensity levels. Each individual illumination intensity level mapping function is 
associated with a defined range of illumination intensity levels. 

The illumination intensity level mapping controller 3, in one embodiment, 

20 determines a plurality of number of pixels, each determined number of pixels 
corresponding to one defined range of illumination intensity levels. In this embodiment, 
the illumination intensity level mapping controller 3 determines a plurality of 
illumination intensity level mapping functions, each determined illumination intensity 
level mapping function corresponding to one defined range of illumination intensity 

25 levels. Each illumination intensity level mapping function is determined based upon the 
determined number of pixels within the associated defined range of illumination intensity 
levels. 

In this embodiment, the transfer control function generation circuit 7 determines a 
transfer control function based on the plurality of determined illumination intensity level 
30 mapping functions and imposes the determined transfer control function upon a pixel of 
the digital imager. 
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In another implementation of the system illustrated in Figure 10, the illumination 
intensity level mapping controller 3 determines a number of saturated pixels and selects a 
first illumination intensity level mapping function when the determined number of 
saturated pixels is above a first threshold. The illumination intensity level mapping 

5 controller 3 determines a number of pixels having illumination intensity levels within a 
defined range of levels and selects a second illumination intensity level mapping function 
when the determined number of pixels is below a second threshold. The transfer control 
function generation circuit 7 determines a transfer control function based on the selected 
illumination intensity level mapping function and imposes the determined transfer control 

1 0 function upon a pixel of the digital imager. 

The transfer control function may comprise a plurality of discrete transfer control 
functions, preferably eight discrete transfer control functions. In this embodiment, the 
determined illumination intensity level mapping function comprises a plurality of discrete 
illumination intensity level mapping functions, preferably eight discrete illumination 

15 intensity level mapping functions. Each discrete transfer control function is determined 
based on one of the plurality of distinct illumination intensity level mapping functions. 

Each discrete illumination intensity level mapping function may be a linear 
illumination intensity level mapping function. The plurality of discrete linear 
illumination intensity level mapping functions form a composite piece-wise linear 

20 illumination intensity level mapping function, the composite piece-wise linear 
compression being the determined illumination intensity level mapping function. The 
determined composite illumination intensity level mapping function is a nearly 
logarithmic illumination intensity level mapping function. 

In a further implementation, the illumination intensity level mapping controller 3 

25 selects a first illumination intensity level mapping function, and the transfer control 
function generation circuit 7 determines a first transfer control function based on the 
selected first illumination intensity level mapping function and imposes the determined 
first transfer control function upon a pixel of the digital imager. The illumination 
intensity level mapping controller 3 then determines a histogram of illumination intensity 

30 levels of pixels of the imaged scene being generated by the digital imager having the 
determined first transfer control function imposed thereon and determines an illumination 
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intensity level maximum, the illumination intensity level maximum representing a 
greatest illumination intensity level for a pixel in a sample forming the histogram. The 
illumination intensity level mapping controller 3 determines a second illumination 
intensity level mapping function, based on the determined intensity level maximum, the 
second illumination intensity level mapping function preventing the generation of any 
saturated pixels and providing a dynamic range of image data enabling each level in the 
histogram to be realized by the digital imager. The transfer control function generation 
circuit 7 then determines a second transfer control function based on the determined 
second illumination intensity level mapping function and imposes the determined second 
transfer control function upon a pixel of the digital imager. The first illumination 
intensity level mapping function represents a maximum compression of the resolution of 
the high illumination intensity levels. 

As noted above, one way of manipulating the transfer control function is to use a 
stepped or piecewise discrete-time transfer control function. By using a stepped or 
piecewise discrete transfer control function, the map of the pixel output voltage versus 
illumination intensity can be modified to enable a wider range of possible illumination 
intensity levels before saturation, while emphasizing low illumination intensity levels. 

However, to use a stepped or piecewise discrete-time transfer control function, 
each stepped transfer control function or control signal level must be determined as well 
as the timing of the transition points between discrete transfer control functions or 

discrete control signal levels. 

In the preferred embodiment of the present invention, the transfer control function 
is a stepped or piecewise discrete-time transfer control function, thereby producing a 
control signal that is a stepped or piecewise discrete-time control signal. 

When the transfer control function generation circuit 7 receives T mt5 the first 
transition point from a first discrete transfer control function to a discrete second transfer 
control function of the composite transfer control function is immediately set. The 
illumination intensity level mapping function, g, determines the actual discrete transfer 
control functions, wherein the actual discrete transfer control functions determine each of 
the actual discrete control signal levels. 
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In the example discussed below, the illumination intensity level mapping 
function, g, is derived from the desired map of output voltage versus illumination 
intensity level of the imaged scene illustrated in Figure 14. As seen in Figure 14, the 
desired map of output voltage versus illumination intensity level of the imaged scene is a 
5 piece-wise discrete mapping, with each discrete mapping corresponding to a different 
compression of the resolution of the associated illumination intensity level range. As 
noted below, each discrete compression of the resolution of the associated illumination 
intensity level range corresponds to a level, b l3 in the transfer control function. As the 
compression of the resolution of the associated illumination intensity level range is 

10 increased, the corresponding level, b b is decreased. 

As noted above, in a preferred embodiment of the present invention, the transfer 
control function is a discrete piece-wise linear function, having eight pieces or steps 
formed by seven transition points, that produces a nearly logarithmic response. 

An example of a stepped or piecewise discrete-time transfer control function, 

15 according to a preferred embodiment of the present invention, is illustrated in Figure 13. 
In Figure 13, the timing of the transition points, t, is set forth on the horizontal axis, and a 
level, b, corresponding to the transfer control function, which corresponds to an amount 
of the subsequently produced control signal being stepped down from a reset level, is set 
forth on the vertical axis. 

20 The stepped or piecewise discrete-time transfer control function in one 

embodiment of the present invention, as illustrated in Figure 13, is realized by: 

1. Setting the slope of region from the first transition point, ti, to the second 

transition point, t 2 , to be m; 

2. Setting the slope of the region from the second transition point, t 2 , to the third 

25 transition point, t 3 , to be gm; 

3. Setting the slope of the region from the third transition point, t 3 , to the fourth 
transition point, U, to be g m; 

4. Setting the slope of the region from the fourth transition point, t 4 , to the fifth 

transition point, ts, to be g m; 
30 5. Setting the slope of the region from the fifth transition point, t 5 , to the sixth 

transition point, \<s, to be g 4 m; and 
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6, Setting the slope of the region from the sixth transition point, t 6 , to the seventh 

transition point, t 7 or T max > to be g 5 m. 

If more than seven transition points are desired, the slope of the region between 
subsequent transition points would continue to increase by a factor g. 

With such an approach, when the exposure controller 5 specifies the desired 
integration time, T int , and the illumination intensity level mapping controller 3 specifies 
the desired illumination intensity level mapping function, g, the stepped transfer control 
function is completely constrained; however, it is still necessary to determine when the 
transition points between the discrete transfer control functions occur during the overall 
frame period, T max . 

As noted above, in this preferred embodiment of the present invention as 
illustrated in Figure 13, there are seven transition points. These seven transition points 
can be determined by solving the resulting sets of simultaneous equations: 

to ~~ T max " T in t 
m = (bi - bo)/T 0 
m g = (b 2 -bi)/Ti 
m g 2 = (b 3 -b 2 )/T 2 
m g 3 = (b 4 -b 3 )/T 3 
m g 4 = (b 5 -b 4 )/T 4 
m g 5 = (b 6 -b 5 )/T 5 
m g 6 - (b 7 -b 6 )/T 6 
T ma x =t 0 +To+T 1 +T 2 +T3+T 4 +T 5 +T 6 

The above equations use the variables all referenced in Figure 13. In a preferred 
embodiment, it is assumed that the levels, b 0 through b 7 , corresponding to the transfer 
control function, which individually correspond to an amount of the subsequently 
produced control signal being stepped down from a reset level, are equally spaced by a 
quantity Ab (except that (b 7 - b 6 = 2Ab), the above equations, solving for To, Ti, T 2 , T 3 , 
T 4 , and T 5 yield 

To = (g 6 )/( g 6 + g 5 + g 4 + g 3 + g 2 + g + 2) * T int 

Ti = (To)/ g 
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T 2 =(T0/g 
T 3 = (T 2 )/ g 
T 4 = (T 3 )/ g 
T 5 = (T 4 )/ g 

With the solving for T 0 , Ti, T 2 , T 3 , T 4 , and T 5 , the seven transition points; to, U, t 2 , 
t 3 , t4, t 5 , and t 6 ; are: 

to — Tmax " T ln t 

ti=to+T 0 
t 2 = ti +Ti 
t 3 =t 2 +T 2 
U= t 3 +T 3 
ts= t 4 +T 4 
fe= t 5 +T 5 

It is noted that the levels, b l9 need not be evenly spaced to find the transition 
points. The equal spacing of b is just an example of creating a transfer control function 
according to the concepts of the present invention. 

In a preferred embodiment of the present invention, the determination of the 
timing of the transition points is realized by dedicated hardware due to the repetitive 
nature of the calculation. Once T 0 is found, one simply does a divide by the same 
denominator five more times. In the preferred embodiment, T 0 is not calculated by 
dedicated hardware due to inefficiencies, thus the number of possible g's is limited and a 

table lookup is used to find T 0 

In this embodiment, the transition points between the plurality of discrete transfer 
control functions, which form a composite transfer control function, are determined 
following the determination of an integration time, T mt5 and an illumination intensity 
level mapping function, the illumination intensity level mapping function providing the 
proper compression of the resolution of the high illumination intensity levels of imaged 
scene and proper emphasis of the low illumination intensity levels of imaged scene. 



-22- 



i I 

In a preferred embodiment, the composite transfer control function has eight 
discrete transfer control functions and seven transition points. The first transition point is 
equal to a difference between a maximum possible integration time, T ma x, or frame 
period, F, and the determined integration time, T int . Subsequent transition points, 
5 assuming that the levels, b l5 are equally spaced by a quantity Ab (except that (b 7 - b 6 = 2 A 
b), are equal to a sum of all previous transition points and a time T s = ((g T1 " 1 )/((g n4+ g n 
2 +. f .+g 2 +g+2)(g (p) )))*T int , where g is equal to the determined illumination intensity level 
mapping function, n is equal to a total number of transition points, p is equal to a 
positional number of the discrete transfer control function being calculated, and T int is 
10 equal to the determined integration time. If the levels, b is are not equally spaced by a 
quantity Ab, the equation would be slightly different depending upon the spatial 
relationship between levels, fy. 

Figure 15 illustrates a process utilized by an illumination intensity mapping 
controller of a preferred embodiment of the present invention. As illustrated in Figure 
15 15, the illumination intensity mapping controller takes the imaged scene from the sensor 
and counts the number of saturated pixels. If there are too many saturated pixels, the 
illumination intensity mapping controller increases the amount of compression upon the 
resolution of the high illumination intensity levels. The illumination intensity mapping 
controller also counts the number of pixels in the upper region (bin) of a histogram of the 
20 imaged scene to determine if the imaged scene fills the entire range of the histogram. If 
there are not enough pixels in the upper region (bin), the illumination intensity mapping 
controller decreases the amount of compression upon the resolution of the high 
illumination intensity levels. The illumination intensity mapping controller continues this 
process until there are no saturated pixels and the histogram is completely used. 
25 This approach works well for video applications in that the illumination intensity 

mapping controller can continuously run without having to drastically change the 
illumination intensity mapping function to determine how to best set it. If the number of 
illumination intensity mapping steps is minimized under the constraint of stability, eight 
illumination intensity mapping steps are sufficient. This allows the illumination intensity 
30 mapping function to be set within 16 frames as a maximum. 

-23- 



Figure 16 illustrates a process utilized by a transfer control function generator 
circuit of a preferred embodiment of the present invention. As illustrated in Figure 16, 
the transfer control function generator circuit sets the initial level, b, of the transfer 
control function to b 0 . A step value, i, is set to a value of 1. The timing of the transition 
point, t s , is calculated. The transfer control function generator circuit goes into a standby 
state, sending the last establish level, b, to the digital imager, until the calculated 
transition point, t„ is realized. When the calculated transition point, t„ is realized, a new 
level, b, is set equal to b,+i. The step value, i, is incremented by 1. The calculating of the 
transition point, t„ and setting of the level, b, is repeated until the step value, i, is equal to 
6. 

It is to be recognized that the present invention contemplates both employing 
preprogrammed illumination intensity level mapping functions as well as computing 
custom illumination intensity level mapping functions on the fly during a given frame 
analysis. The use of preprogrammed illumination intensity level mapping functions can 
be preferred for applications in which computational power is to be minimized. Real 
time computation of illumination intensity level mapping functions can be preferred for 
applications in which flexibility in sensitivity control is desired. 

For example, given a frame analysis in which a large number of pixels have a 
particular intensity level, a customized illumination intensity level mapping function 
computed to allocate a large number of output levels to that common intensity level 
would be optimal for efficiently parceling the output range levels. Similarly, 
computation of a customized illumination intensity level mapping function can be 
optimal for allocating a large number of output levels to the intensity range of a particular 
physical region of pixels of an image frame. Such a region may be of interest for its 
inclusion of, e.g., moving objects, or in general, objects of interest or importance in a 
scene. The present invention further contemplates use of both preprogrammed and 
custom, on-the-fly illumination intensity level mapping functions for a single application, 
whereby the benefits of both approaches can be obtained. 

The adaptive sensitivity control, on a pixel-by-pixel basis, of the present invention 
preferably is implemented to operate automatically, without the necessity for user control 
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or input, to enable automatic, real time, adaptive adjustment of the digital imager for 
maximizing image quality. 

Figure 1 1 illustrates another embodiment of the present invention. As illustrated 
in Figure 11, in a first step, the integration time, T int , of the digital imager is controlled. 
Integration time control is specifically implemented by controlling the number of pixels 
associated with low illumination intensity levels. This has the effect of shifting the 
dynamic range of the digital imager relative to the scene illumination intensity range. 

In one example, intensity levels 0 through 3 are defined as a specified low 
intensity level range. Given that the low intensity level range includes levels 0 through 3, 
this accounts for 0.1% of the total number of intensity levels of a 4,096 output intensity 
levels imager. The integration time is controlled in accordance with the present invention 
so that the same percentage, 0.1%, of the total number of pixels, i.e., 0.1% of a 307,200 
pixel imager, or 307 pixels, have the intensity levels of 0, 1, 2, or 3. This acts to enforce 
a condition whereby the entire 4096 levels are efficiently parceled across the intensity 
range of the scene. 

Preferably, the number of pixels selected to be associated with the low intensity 
level range is defined as a range; e.g., in the example just given, 307 +/- 30 pixels are 
preferably assigned to the range. The use of a number range of pixels enables the 
introduction of hysteresis, which stabilizes the system. Another way to introduce the 
hysteresis is to use an intensity level range. For example, if the number of pixels at the 
intensity levels of 0 or 1 is less than 307, in response, the exposure time can be decreased. 
Similarly, if the number of pixels at the intensity levels of 0, 1, 2, 3, or 4 is more than 307, 
then the exposure time can be increased. In this example, the threshold value for the 
number of pixels is 307 and does not have any ranges, but the intensity level itself has a 
range from "0 or 1" to "0, 1, 2, 3, or 4". This introduction of intensity level ranges also 
provides hysteresis. The low intensity level range is preferable small, and typically the 
smaller the better, except that if too small, e.g., including only the 0 level, the algorithm 
may not be robust to noise levels or to the occurrence of faulty pixels. 

In the first step of Figure 11, the number of pixels of the frame that are found to 
be associated with the defined low intensity ranges are counted from the frame 
illumination intensity histogram. If the number of low intensity range pixels is larger 
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than that number corresponding to the percentage of low intensity ranges as a function of 
the total number of intensity levels, the integration time of the digital imager is increased. 
Conversely, if the number of pixels is smaller, the digital imager integration time is 
decreased. 

5 In step 2 of Figure 11, the illumination intensity level mapping function is 

adjusted. This is specifically realized by selecting an illumination intensity level 
mapping function to make a correspondence between a number of pixels and a high 
illumination intensity range. This correspondence is particularly made so that the 
distribution of pixels goes to zero at the saturation level of the digital imager, as shown in 

10 Figure 9. The high illumination intensity range can be defined as, e.g., the intensity 
levels 4,095, 4,094, 4,093, and 4,092, given 4096 output levels. For this example, the 
number of pixels to be associated with these output levels is then .1% of the total number 
of pixels. This pixel number can then be defined as the range between 277 and 337 
pixels, for example, for a 307,200 pixel imager. 

15 With this step completed, the transfer control function of the digital imager is 

adjusted such that the digital imager output levels are adjusted in the manner illustrated in 
Figure 9. It is recognized in accordance with the present invention that the step of 
adjustment of imager integration time can be carried out after or in parallel with the step 
of adjustment of the level of digital imager control signal. The order of steps shown in 

20 the flowchart of Figure 1 1 is an example only. 

A simple example is described below to illustrate the concepts of the present 

invention. 

Consider an example digital imager having a pixel array size of 4x4, where for a 
given scene being imaged in a frame under analysis, all the pixels in each row have the 
25 same intensity level, and where the intensity level of each pixel is output in 4-bits ranging 
in output levels between 0 and 15. 

Given further an example histogram of scene intensity levels to be imaged by the 
particular pixels of the digital imager is as follows: 

30 3 for the first pixel row, 

6 for the second pixel row 
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300 for the third pixel row, and 
600 for the fourth pixel row. 

In other words, the scene intensity level ratios between the pixel rows are: 

5 

1 :2 for the first and second pixel rows, 
1 :50 for the second and third pixel rows, and 
1 :2 for the third and fourth pixel rows. 

10 In a conventional sensitivity control technique, in order to reduce the intensity 

y — V. 

% level for the fourth row from 600 to 15, which is the highest intensity level capable of the 

- *> 

W digital imager, the intensity of each pixel would need to be reduced by a factor of 40 by 

ffl decreasing the integration time, with the new intensities after the integration time 

l' n adjustment given as: 

« > m *r 
»* 

<<H' ~ 

Q 

i2 0 (=1/40) for the first pixel row, 

W 0 (=2/40) for the second pixel row, 

p 8 (=300/40) for the third pixel row, and 

a y 1 5 (=600/40) for the fourth pixel row. 

20 

Note that this results in a loss of the original intensity difference between the first 
and second pixel rows, and therefore results in a poor output image. 

This deficiency is overcome by the technique of the present invention. Following 
the technique of the present invention described above, the intensities of the pixels are 

25 adjusted as follows. 

(a) Each illumination intensity level is divided by a factor of 3, by reducing the 

integration time, in order to bring the lowest illumination intensity level of the scene 

being imaged to 1 , the designated low illumination intensity level of the digital imager. 

The new illumination intensity output level of pixel rows one-four are then given as 

30 1 (=3/3), 2(=6/3), 100 (=300/3), and 200 (=600/3), respectively. 
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(b) Each illumination intensity level is subjected to a non-linear illumination 
intensity level mapping function. The illumination intensity levels of 1 and 2 stay 
substantially the same, but the resolution of the highest illumination intensity level pixel 
region, having the intensity level 200, is compressed strongly to map highest illumination 

5 intensity level of the scene being imaged to the maximum intensity level 15 of the digital 
imager. The resolution of the high illumination intensity level pixel region, just below 
the highest illumination intensity level pixel region, is compressed not as strongly, and 
therefore the illumination intensity level of the third pixel row is set at 9. 

With these steps, the technique of the present invention converts the original pixel 

10 row illumination intensity level distribution of 1, 2, 300, and 600 from the scene into 
imager output levels of 1, 2, 9, 15. The original condition in which the four pixel rows 
have different illumination intensity levels is preserved, while the image illumination 
range is fully mapped to the output range characteristic of the digital imager. 

The present invention contemplates a wide range of implementation alternatives 

15 for accommodating various operating considerations. For example, the computational 
load of the method described above can be relatively large due to the use of an 
illumination intensity level histogram calculation. A number of alternative methods can 
be employed to reduce the computational load, each at some cost of performance 
degradation. It is found, however, that the most prominent performance degradation is 

20 generally manifested only as a longer settling time when the illumination intensity of a 
scene is changing significantly; such is typically considered to be an acceptable condition. 

In a first example alternative technique, the number of illumination intensity 
levels of the histogram calculation is reduced. For example, 4,096 original illumination 
intensity levels can be replaced with a smaller number, e.g., four, illumination intensity 

25 level groups. The illumination intensity level groups 1, 2, 3, and 4 each can be mapped 
to, e.g., illumination intensity levels of 0-523, 524-1,047, 1048-1,571, and 1,572-4,095, 
respectively. This grouping of illumination intensity levels significantly reduces the 
needed computational load. 

In a further alternative technique, the histogram calculation can be eliminated. In 

30 this technique, the pixel illumination intensity distribution is specified to be even across 
4095 levels. In this case, 75 pixels are designated for each illumination intensity level for 
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a 640x480 pixel array, because 640x480/4,095=75. Therefore, 150 (=75x2) pixels are 
designated for the illumination intensity levels 0 and 1 of a low illumination intensity 
range. 

Figure 12, illustrates a further embodiment of the present invention for 
5 establishing the integration time. As shown in Figure 12, the total number of pixels for 
the illumination intensity levels 0 and 1 are counted, and noted as NumberDark (ND). 
After determining ND, the process determines the adjustment to the integration number 
to reduce the value of ND. The method for increasing the integration time to reduce ND 
is as follows: 

|s& 10 

!j (1) If 165<ND<=180, increase the integration time by 10%; 

(2) If 1 80<ND<=195, increase the integration time by 20%; 
m (3) If 1 95<ND<=2 1 0, increase the integration time by 30%; 

$j (4) If 210<ND<=225, increase the integration time by 40%; 

?, : i 

* 1 5 (5) If 225<ND<=240, increase the integration time by 50%; 

£ (6) If 240<ND<=270, increase the integration time by 70%; 

z 

O (7) If 270<ND<=300, increase the integration time by 90%; 

p (8) If 300<ND<=450 ? increase the integration time by 150%; 

*TLf* il 
i. - 

1} " ** 

2 y (9) If 450<iND<=600, increase the integration time by 250%; 

20 (10) If 600<ND<=900, increase the integration time by 400%; 

(1 1) If 900<ND<=1500, increase the integration time by 750%; 

(12) If 1500<ND, increase the integration time by 1 ? 000%; and 

If ND is between 135 and 165, there is no change in integration time. 



25 If ND is less than 135, the following rules are instead applied to decrease the 

integration time: 

(1) If 120<ND<=135, decrease the integration time by 11%; 

(2) If 105<ND<=120, decrease the integration time by 25%; 
30 (3) If 90<ND<=105, decrease the integration time by 42%; 

(4) If 75<ND<=90, decrease the integration time by 66%; 
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(5) If 60<ND<=75, decrease the integration time by 100%; 

(6) If 30<ND<=60, decrease the integration time by 233%; 

(7) If 25<ND<=30, decrease the integration time by 400%; 

(8) If 20<ND<=25, decrease the integration time by 500%; 
(9) If 15<ND<=20, decrease the integration time by 650%; and 

(10) If ND<=15, decrease the integration time by 1 ? 000%. 

Before, after, or simultaneous with this integration time control, the illumination 
intensity level mapping function of the digital imager is controlled so that the number of 
pixels that have the illumination intensity levels of either 4095 or 4094 is controlled to be 
between 135 and 165. The number of such "bright" pixels, high illumination intensity 
level pixels, is counted and designated NumberBright (NB), and then the illumination 
intensity level mapping function is adjusted accordingly. In one example, the 
illumination intensity level mapping function is selected from a number, e.g., eight, 
different illumination intensity level mapping functions available. It is contemplated in 
accordance with the present invention that any reasonable number of functions can be 
provided. 

In this embodiment, the following rules are applied, based on the counted NB, to 
increase or decrease the compression of the resolution of the high illumination intensity 
level through the illumination intensity level mapping function of the digital imager: 

(1) If 165<NB<=200, invoke a stronger compression upon the resolution 
of the high illumination intensity levels by one notch. 

(2) If 200<NB<=300, invoke a stronger compression upon the resolution 
of the high illumination intensity levels by two notches. 

(3) If 300<NB<=400, invoke a stronger compression upon the resolution 
of the high illumination intensity levels by three notches. 

(4) If 400<NB<=500, invoke a stronger compression upon the resolution 
of the high illumination intensity levels by four notches. 

(5) If 500<NB<=600, invoke a stronger compression upon the resolution 
of the high illumination intensity levels by five notches. 
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(6) If 600<NB<=700, invoke a stronger compression upon the resolution 
of the high illumination intensity levels by six notches. 

(7) If 700<NB, invoke a stronger compression upon the resolution of the 
high illumination intensity levels by seven notches. 

(8) If 120<NB<=135, invoke a weaker compression upon the resolution of 
the high illumination intensity levels by one notch. 

(9) If 100<NB<=120, invoke a weaker compression upon the resolution of 
the high illumination intensity levels by two notches. 

(10) If 80<NB<=100, invoke a weaker compression upon the resolution of 
the high illumination intensity levels by three notches. 

(11) If 60<NB<=80, invoke a weaker compression upon the resolution of 
the high illumination intensity levels by four notches. 

(12) If 400<NB<=60, invoke a weaker compression upon the resolution of 
the high illumination intensity levels by five notches. 

(13) If 20<NB<=40, invoke a weaker compression upon the resolution of 
the high illumination intensity levels by six notches. 

(14) If 20>NB, invoke a weaker compression upon the resolution of the 
high illumination intensity levels by seven notches. 

These rules enable selection of an illumination intensity level mapping function that 
proportions the intensity levels of the pixels in the manner described earlier. 

In one embodiment of the present invention, a number of pixels of image data 
having illumination intensity levels within a first defined range of illumination intensity 
levels is determined. From this determined number of pixels, an illumination intensity 
level mapping function is determined that is used to create a transfer control function that 
can then be imposed upon a pixel of the digital imager. A number of pixels having 
illumination intensity levels within a second defined range of illumination intensity levels 
is determined to enabling the determination of an integration time. If the integration 
function is determined, the determination of the transfer control function is based on the 
determined illumination intensity level mapping function and the determined integration 
time. 
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The determination of an illumination intensity level mapping function and 
integration time is repeated until a desired dynamic range is realized. The first defined 
range of illumination intensity levels is a range of illumination intensity levels including 
an illumination intensity level representing pixel saturation, and the second defined range 
5 of illumination intensity levels is a range of illumination intensity levels including an 
illumination intensity level representing a minimum illumination intensity level, wherein 
the minimum illumination intensity level can be adjusted for a pixel offset, residual pixel 
charge. 

The determination of a number of pixels of image data having illumination 
10 intensity levels within a first defined range of illumination intensity levels can be from a 
frame of pixels of image data created by the digital imager, from a partial frame of pixels 
of image data created by the digital imager, from a defined area within a frame of pixels 
of image data created by the digital imager, or from a user-defined area within a frame of 
pixels of image data created by the digital imager. 
15 The illumination intensity level mapping function can be either calculated or 

selected from a plurality of pre-specified illumination intensity level mapping functions, 
preferably from eight pre-specified illumination intensity level mapping functions, based 
upon the determined number of pixels within the first defined range of illumination 
intensity levels. Moreover, the transfer control function can be either calculated or 
20 selected from a plurality of pre-specified transfer control functions based upon the 
determined illumination intensity level mapping function and determined integration 
time. 

Furthermore, the determination of the transfer control function can realized during 
the generation of a second frame of image data that is immediate subsequent to the frame 
25 of image data being used to make the determination. In this situation, the new transfer 
control function is imposed upon pixels during the generation of a third frame of image 
data, the third frame being immediate subsequent to the second frame of image data. 

On the other hand, if the determination is being made on only a partial frame, the 
determination of the transfer control function can realized during the generation of a first 
30 frame of image data which is the frame of image data having the partial frame of image. 
In this situation, the new transfer control function is imposed upon pixels during the 
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generation of a second frame of image data, the second frame being immediate 
subsequent to the first frame of image data. 

The illumination intensity level mapping function may be determined independent 
of, dependent upon, after, before, or substantially simultaneous with the determination of 
the integration time. Moreover, the illumination intensity level mapping function may be 
a composite function comprising individual discrete functions wherein each individual 
discrete illumination intensity level mapping function is determined based upon the 
determination of a number of pixels within a defined range of illumination intensity 
levels. Each individual illumination intensity level mapping function is associated with a 
defined range of illumination intensity levels. 

In another embodiment of the present invention, a number of saturated pixels is 
determined and a first illumination intensity level mapping function is selected when the 
determined number of saturated pixels is above a first threshold. This embodiment also 
determines a number of pixels having illumination intensity levels within a defined range 
of levels and selects a second illumination intensity level mapping function when the 
determined number of pixels is below a second threshold. The transfer control function 
is determined based on the selected illumination intensity level mapping function and 
imposed upon a pixel of the digital imager. 

In a third embodiment, the present invention determines a number of pixels of 
image data having illumination intensity levels within a first defined range of 
illumination intensity levels, the first defined range of illumination intensity levels 
including an illumination intensity level corresponding to a pixel saturation value and 
determines an illumination intensity level mapping function based thereupon. An 
integration time is determined based upon a number of pixels having illumination 
intensity levels within a second defined range of illumination intensity levels, the second 
defined range of illumination intensity levels including an illumination intensity level 
corresponding to a minimum illumination intensity level. A transfer control function is 
created based upon the determined illumination intensity level mapping function and the 
determined integration time and imposed upon a pixel of the digital imager. 

The transfer control function comprises a plurality of discrete transfer control 
functions, preferably eight discrete transfer control functions. 
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In a further embodiment, the present invention selects a first illumination intensity 
level mapping function and determines a first transfer control function based on the 
selected first illumination intensity level mapping function. The determined transfer 
control function is imposed upon a pixel of the digital imager. Thereafter, a histogram of 
illumination intensity levels of pixels of image data being generated by the digital imager 
having the determined first transfer control function imposed thereon is determined and 
an illumination intensity level maximum is determined, the illumination intensity level 
maximum representing a greatest illumination intensity level for a pixel in a sample 
forming the histogram. A second illumination intensity level mapping function is 
determined based on the determined intensity level maximum. The second illumination 
intensity level mapping function prevents the generation of any saturated pixels and 
provides a dynamic range of image data enabling each level in the histogram to be 
realized by the digital imager. A second transfer control function is determined based on 
the second illumination intensity level mapping function and imposed upon a pixel of the 
digital imager. The first illumination intensity level mapping function is a maximum 
compression of the resolution of the high illumination intensity levels of the imaged 
scene. 

The present invention contemplates additional techniques for adaptively 
controlling the sensitivity, on a pixel-by-pixel basis, of a digital camera. For example, 
the integration time control techniques described above can be replaced or combined with 
any suitable mechanical iris control and/or intensity signal gain control. Mechanical iris 
control and intensity signal gain control can be employed to produce results similar to the 
integration control, but these controls produce some side effects also. If the mechanical 
iris is increased, for example, the depth of focus will be correspondingly decreased. If 
the signal gain is increased, as another example, the noise level, as well as the signal level 
will be correspondingly increased. 

The description provided above and accompanying diagrams are provided to 
demonstrate the broad scope of the adaptive sensitivity control contemplated by the 
invention. No particular implementation of the techniques of the invention is required; 
rather, any implementation that enables compression rate and integration time control of 
a digital imager to enable imager sensitivity control is contemplated by the invention. 
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While various examples and embodiments of the present invention have been 
shown and described, it will be appreciated by those skilled in the art that the spirit and 
scope of the present invention are not limited to the specific description and drawings 
herein, but extend to various modifications and changes all as set forth in the following 
5 claims. 
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